The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
Parking spaces are frequently in short supply and high in demand. Accordingly, cities and towns will often provide metered parking spaces and handicapped parking spaces. Handicapped spaces are designated only for authorized handicapped persons. The parking meters accept coins or dollar bills in return for time on the meter, which represents the amount of authorized time that the vehicle may remain within the parking space. If the vehicle remains within the parking space after the meter expires, the owner of the parked vehicle is subject to citation or towing. Alternatively, in many other parts of the city, the parking garages/parking lots would provide a sheltered or open aired parking space. Sometimes these are multi-level garages, to support multiple cars. In an exemplary sub-set the parking lot is claimed by the authorities at a space for the convenience of the public and parking attendants are commissioned/enstated to facilitate parking for shoppers, show owners, visitors etc. In residential and business establishment, the location within the driveway or underground/basement levels and the side-walk location adjoining the establishment are granted to allow for parking.
Private parking spaces are also available in areas where parking is in short supply. These parking spaces typically come at some expense to the vehicle owner, but usually offer the convenience of not requiring that the owner pay money throughout the day. Moreover, private parking spaces often offer greater security and a convenient location. Many private garages issue a fixed number of monthly parking passes sometimes costing $200.00-$300.00 or more. Unfortunately, unauthorized persons often park in the assigned spaces resulting in significant inconvenience to both the owner of the private parking lot and the owner of the vehicle which is entitled to the parking space. In order to reduce the unauthorized parking within the assigned spaces, the parking lot owners will often employ expensive attendants to monitor the parking lot.
The standard practice of issuing parking violations involves the use of officials, commonly referred to as “meter-maids”. These officials scour the streets in search of violators. Once a violation is located, the official exits their vehicle, if not traveling on foot, and records important vehicle information. The officer then writes out a written acknowledgment of the violation, referred to as a parking citation or ticket. Unfortunately, this process requires considerable time to search for and record violations. This time is costly and could be more productively used. In addition, parking violation officials frequently find numerous vehicles which are simultaneously violating parking laws. While the official is preparing one written citation, owners of other vehicles exit the scene before a violation can be issued. This also results in loss of money to the city and ability for risk taking by defaulters, knowledgeable of their ability to “get-away”. In other cases, the parking enforcement car drives around the neighborhood and determines if a parking spot is empty or occupied and then the information fed to the central server from the parking pay stations about the expiration of parking times for various spots.
A paper titled “License Plate Detection and Recognition Using Deeply Learned Convolutional Neural Networks” by Syed Zain Masood, et al, proposes a sighthound's fully automated license no. plate detection and recognition system. The core technology of the system is built using a sequence of deep Convolutional Neural Networks interlaced with accurate and efficient algorithms.
A paper titled “Real-Time Illegal Parking Detection System Based on Deep Learning” by Xuemei Xie et al, proposes a novel illegal vehicle parking detection system. Illegal vehicles captured by camera are firstly located and classified by the famous Single Shot Multi-Box Detector (SSD) algorithm. After that, a tracking and analysis of movement is adopted to judge the illegal vehicles in the region of interest.
United States Patent Publication 20170017848A1 filed by Magna Electronics Inc. teaches a parking assist system of a vehicle that includes a camera that, when disposed at the vehicle, has a field of view exterior of the vehicle. An image processor is operable to process image data captured by the camera to detect parking space markers indicative of a parking space and to identify empty or available parking spaces.
Indian Patent Publication 201711020147A filed by HCL Technologies Limited teaches a driver of an automobile for parking the automobile. The system may detect multiple parameters associated with an automobile, and preferences associated with the driver and allocate a parking by generating a parking token corresponding to the parking slot and provide a first set of navigation instruction and orientation instructions for guiding the driver to reach the parking slot based on monitoring of the parking token.
There are other violations that have been determined for drivers of vehicles for example, not wearing helmets, which is a hazard for bike/scooter riders and is considered unlawful by many, speeding is another most dangerous violation on the roads and many cities impose heavy penalties including revocation of license to drive and other civil actions against drivers over speeding or abetting over speeding. Also, road lanes are made and demarcated for drivers to maintain their lanes while driving for safe commuting and safe maneuvering across lanes, merging into traffic, overtaking etc. lane departure without notification is a serious jeopardy to other motorists and creates aggressive responses from others who are following the lanes, due to the dangers involved in such type of driving. Further, jumping red lights or stop signs are considered very severe violations of traffic, posing risk of life and property for everyone within reach of the violation area. Further, other multiple violations are also important to record and cause grave harm to people and property. Examples of some available prior-art literature are as listed below:
A paper titled “a system for tracking and annotating illegally parked vehicles, by Vrusias et. al., presents an automatic method for identifying such events in CCTV video, by first tracking all the related objects and then annotating the events with appropriate keywords for storing and retrieval purposes. The method proposed makes use of a combination of video object tracking algorithms and techniques for capturing knowledge in keyword-based ontology structures. Starting from low level visual information extracted from each video frame, high-level semantics such as moving objects are identified and classified. A probabilistic model, which takes its inputs from the visual modules, is used for identifying illegally parked vehicles. Finally, the keyword ontology, constructed automatically from expert descriptions, is linked to objects identified in the video.
A paper titled “Police Eyes” (DOI: 10.11.09/ECTICon.2013.6559635, Print ISBN: 978-1-4799-0546-1, Publisher: IEEE, Conference Location: Krabi, Thailand) describes a mobile, real-time traffic surveillance system we have developed to enable automatic detection of traffic violations. Police Eyes would be useful to police for enforcing traffic laws and would also increase compliance with traffic laws even in the absence of police. The system detects illegal crossings of solid lines using image processing and efficient computer vision techniques on image sequences acquired from static IP cameras.
Chinese Patent application CN101183427A by Chengjun is system that checks the violation of illegal parking by monitoring a parking region continuously from a video stream acquired by an omnidirectional sensor. When a vehicle enters the monitored region, the system starts an event which comprises of tracking the vehicle using a multi-object tracker, license plate recognition using background subtraction and template matching. This patent doesn't employ any Deep learning methods.
A paper titled “Signal Jump Detection Process” (International Journal of Computer Applications Technology and Research, Volume 6—Issue 2, 101-105, 2017, ISSN:-2319-8656), provides a system employing image processing techniques namely edge detection and image segmentation in order to detect and recognize the number plate of the vehicle which jumps a red light at a traffic signal.
A paper titled “Integrating motion and appearance for overtaking vehicle detection (Published in: Intelligent Vehicles Symposium Proceedings”, 2014 IEEE, DOI: 10.1109/IVS.2014.6856598,), proposes an algorithm for detecting overtaking vehicles using motion cues from the scene. Motion compensation of video data is performed using the optical flow of the scene and epipolar geometry. After post processing and outlier removal, overtaking vehicle candidates are produced.
A paper titled “Overtaking vehicle detection using a spatio-temporal CRF” (Published in: Intelligent Vehicles Symposium Proceedings, 2014 IEEE, DOI: 10.1109/IVS.2014.6856546), provides a novel CRF model to make use of the interaction between local regions, and the motion features from multiple scales as well. The whole model is based on the low-level optical flows.
A paper by Zhu, et al., titled “Traffic-sign detection and classification in the wild”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2110-2118), provides promising results that have been achieved in the areas of traffic-sign detection and classification
U.S. Pat. No. 6,546,119B2, “Automated traffic violation monitoring and reporting system” by Redflex Traffic Systems provides a system for monitoring and reporting incidences of traffic violations at a traffic location is disclosed. The system comprises a digital camera system deployed at a traffic location. The camera system is remotely coupled to a data processing system. The mounted camera system is used for traffic light violation detection. For the task of red light violation the intersection camera system is strategically placed at an intersection to monitor and record incidences of drivers disobeying a red light. When a vehicle is detected approaching the stop line of a monitored lane, it is tracked and its speed is calculated. If the vehicle is detected entering the intersection against the traffic signal, an evidentiary image set is captured.
A paper by Huang, Shih-Shinh, et al. titled “On-board vision system for lane recognition and front-vehicle detection to enhance driver's awareness, Robotics and Automation, 2004 Proceedings ICRA'04 2004 IEEE International Conference on. Vol. 3. IEEE, 2004, provides a detecting and warning system which is able to pick up the information about two most familiar on-road objects: lane and vehicle.
A paper by Gurghian, et. al., titled “DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks: in CVPR Workshops (pp. 38-45) presents an approach to estimate lane positions directly using a deep neural network that operates on images from laterally-mounted down-facing cameras. To create a diverse training set, they present a method to generate semi-artificial images.
U.S. Pat. No. 9,286,524B titled “Multi-task deep convolutional neural networks for efficient and robust traffic lane detection” assigned to University of Technology Sydney, Toyota Motor Corp provides a computing device comprising of one or more processors for controlling operations of the computing device; and a memory storing data and program instructions used by the one or more processors,
A paper by Angelova, Anelia, et al. titled “Real-Time Pedestrian Detection with Deep Network Cascades” presents a new real-time approach to object detection that exploits the efficiency of cascade classifiers with the accuracy of deep neural networks. Deep networks have been shown to excel at classification tasks, and their ability to operate on raw pixel input without the need to design special features is very appealing. However, deep nets are notoriously slow at inference time. In this paper, the authors propose an approach that cascades deep nets and fast features, which is both very fast and very accurate. They apply it to the challenging task of pedestrian detection. Our algorithm runs in real-time at 15 frames per second. The resulting approach achieves a 26.2% average miss rate on the Caltech Pedestrian detection benchmark, which is competitive with the very best reported results. It is one of the first work that achieves very high accuracy while running in real-time.
A paper by Xie, Xuemei, et al. titled “Real-Time Illegal Parking Detection System Based on Deep Learning” proceedings of the 2017 International Conference on Deep Learning Technologies, A C M, 2017. DOI: 10.1145/3094243.3094261, provides a novel illegal vehicle parking detection system. Illegal vehicles captured by camera are firstly located and classified by the famous Single Shot MultiBox Detector (SSD) algorithm. After that, a tracking and analysis of movement is adopted to judge the illegal vehicles in the region of interest (ROI).
US patent application US20170300763A1 titled “Road feature detection using a vehicle camera system” by GM Global Technology Operations LLC, provides a computer-implemented method for road feature detection, the method comprising receiving, by a processing device, an image from a camera system associated with a vehicle on a road; generating, by the processing device, a top view of the road based at least in part on the image; detecting, by the processing device, lane boundaries of a lane of the road based at least in part on the top view of the road; and detecting, by the processing device, a road feature within the lane boundaries of the lane of the road using machine learning.
A paper by Zeng, Xingyu, et al. titled “Deep learning of scene-specific classifier for pedestrian detection” provides performance of a detector depends much on its training dataset and drops significantly when the detector is applied to a new scene due to the large variations between the source training dataset and the target scene. In order to bridge this appearance gap, the authors propose a deep model to automatically learn scene-specific features and visual patterns in static video surveillance without any manual labels from the target scene. They also propose a cluster layer in the deep model that utilizes the scene specific visual patterns for pedestrian detection.
A paper by Haloi, Mrinal et al, titled “A robust lane detection and departure warning system.” is based on single camera sensor. For lane detection a modified Inverse Perspective Mapping using only a few extrinsic camera parameters and illuminant Invariant techniques is used. Lane markings are represented using a combination of 2nd and 4th order steerable filters, robust to shadowing. Effect of shadowing and extra sun light is removed using Lab color space, and illuminant invariant representation. Lanes are assumed to be cubic curves and fitted using robust RANSAC. This method can reliably detect lanes of the road and its boundary. This method has been experimented in Indian road conditions under different challenging situations and the result obtained was very good. For lane departure angle an optical flow based method were used.
US patent application US20170372161A1 titled “intelligent automatic license plate recognition for electronic tolling environments” by Accenture implements technical solutions that improve the accuracy of automatic license plate recognition. The IALPR analyzes an image of a vehicle proximate to a toll collection point using optical character recognition (OCR), and determines candidate license plate identifications based, at least in part, on the corresponding OCR confidence level. The IALPR can also perform fingerprinting for candidate license plate images and matching analysis with a knowledge base, resulting in additional confidence levels. The IALPR can also perform behavioral analysis on the candidate license plate identifications, including trip context analysis, historical behavioral analysis, or other analytics. The IALPR can generate an overall confidence level for the candidate license plate identifications responsive to the OCR and vehicle fingerprint confidence levels and the behavioral analysis. This enhanced analysis helps the IALPR reduce the number of incorrect license plate identifications and reduce the need for human review.
U.S. Pat. No. 8,509,486B2 titled “Vehicle license plate recognition method and system thereof” by National Chiao Tung University requires a region where a vehicle license plate image exists is detected according to the edge densities of an input image and a vehicle license plate specification. A text area of the vehicle license plate image is divided into a plurality of character images. The character images are binarized to obtain a plurality of binarized character images. A plurality of characters is recognized from the binarized character images. The characters are recombined to form a character string. The abovementioned steps are repeated to obtain a new character string from another image of the same vehicle, which is captured at a next time point. The character string is compared with the new character string character by character to obtain a comparison result for verifying reliability of recognition through a voting technique.
U.S. Pat. No. 8,184,863B2 titled “Video speed detection system” by American Traffic Solutions Inc. provides a system and method for measuring vehicle speeds using video sensing. The system comprises a tracking camera that generates accurately time-stamped video sequences of the monitored road at high frame rates and a processing unit that analyzes the video sequences in real time to detect moving vehicles and calculate their speeds using the time-over-distance method. The system automatically detects moving vehicles in each image frame and derives vehicle positions from a projective mapping established from reference markers on the road. Time information is obtained from the date and time stamp associated with each image frame. In one example, the system also allows a user of the system to set a trigger speed, and the system automatically saves a video clip of the vehicle whose speed exceeds the trigger speed, recording the speeding vehicle passing through the monitored traffic zone defined by the reference markers.
Chinese patent CN202422420U titled “Illegal parking detection system based on video monitoring by Dalian University for Nationalities” provides a utility model discloses an illegal parking detection system based on video monitoring. The illegal parking detection system comprises an image collector, an image processor, an alarm system and a display, wherein the image collector is used for collecting a video image and outputting a video sequence; the image processor is used for carrying out background modeling on the video sequence collected by the image collector by utilizing a codebook model, obtaining a foreground likelihood information image by adopting a background subtraction method.
US patent application US20150235091A1 titled “Lane-line recognition apparatus by Denso Corp” provides an apparatus for recognizing a lane line, comprising of an edge-point extractor configured to extract edge points in an image of surroundings of a subject vehicle including a roadway ahead of the subject vehicle
U.S. Pat. No. 9,305,223B1 titled “Vision-based indicator signal detection uses spatiotemporal filtering” by Google LLC provides an autonomous vehicle is configured to detect an active turn signal indicator on another vehicle. An image-capture device of the autonomous vehicle captures an image of a field of view of the autonomous vehicle. The autonomous vehicle captures the image with a short exposure to emphasize objects having brightness above a threshold.
EP patent application EP2487454A2 titled “Vehicle length estimation” by Scania CV AB describes a method for determining the configuration of a moving vehicle The invention comprises also a system for determining the configuration of a moving.
EP patent application EP2574958A1 titled “Road-terrain detection method and system for driver assistance systems” by Honda Research Institute Europe GmbH describes a road terrain detection system that comprises a method for classifying selected locations in the environment of a based on sensory input signals such as pixel values of a camera image. The method according to any of the preceding claims, wherein the method applied for a specific road terrain such as “road-like area”, “drivable road”, “ego-lane”, “non-ego-lane”, “non-drivable road”, “sidewalk”, “traffic island”, or “off-limits terrain” is automatically parameterized by using positive and negative samples, which are given by training regions such as polygons.
EP patent application EP2578464A1 titled “Video-based warning system for a vehicle” by Honda Research Institute Europe GMBH describes a warning system that can be implemented in any kind of, in order to efficiently detect moving objects. The system utilizes at least one camera for a continuous imaging of the surroundings of the vehicle. Thereby, moving objects can be monitored. A computing unit is programmed to estimate a motion of any moving object based on a pixel motion in the camera image. If a dangerously moving object is detected, a warning unit can be used for issuing a warning signal.
EP patent application EP2863338A2 titled “Delayed vehicle identification for privacy enforcement” by Xerox Corp describes a method for recognition of an identifier such as a license plate includes storing first visual signatures, each extracted from a first image of a respective object, such as a vehicle, captured at a first location, and first information associated with the first captured image, such as a time stamp. Then a second visual signature is extracted from a second image at different times and locations
EP patent application EP2966590A1 titled Lane level traffic by Here Global BV describes a method in which lane level traffic levels are determined based on traffic camera images. A controller aligns a three-dimensional map with a traffic camera view, and identifies multiple lanes in the traffic camera view based on lane delineations of the three-dimensional map.
WO patent application WO2015056105A1 titled “Forward-facing multi-imaging system for navigating a vehicle” by Mobileye Vision Technologies describes a system and method which use cameras to provide autonomous navigation features. In one implementation, a driver-assist system is provided for a vehicle.
WO patent application WO2013009697A1 titled “Image-based vehicle detection and distance measuring method and apparatus” by Bendix Commercial vehicle systems describes an image based vehicle detection and measuring apparatus that employs an accelerated, structured, search method is described for quickly finding the extrema of a multivariable function.
WO patent application WO2014130178A1 titled “A method to detect nearby aggressive drivers and adjust driving modes” by Google Inc. describes a computing device may be configured to receive sensor information indicative of respective characteristics of vehicles on a road of travel of a first vehicle. The computing device may be configured to identify, based on the respective characteristics, a second vehicle that exhibits an aggressive driving behavior manifested as an unsafe or unlawful driving action. Also, based on the respective characteristics, the computing device may be configured to determine a type of the second vehicle. The computing device may be configured to estimate a distance between the first vehicle and the second vehicle.
EP patent application EP3038012A1 titled “Signal for identifying traffic lights for computer vision” by Kaparazoom SLU describes a method for identifying traffic lights. The invention relates to a signal for identifying traffic lights for pedestrians, using an electronic urban guidance device with computer vision, for blind or visually impaired people. Said identification signal can be used to identify other types of urban elements.
U.S. Pat. No. 9,779,314B1 titled “Vision-based detection and classification of traffic lights” by Waymo LLC describes an autonomous vehicle having a vehicle control system. The vehicle control system includes an image processing system. with the image processing system providing instructions to control the autonomous vehicle based on the particular candidate portion representing an illuminated component of a traffic light.
U.S. Pat. No. 9,428,192B2 titled “Vision system for vehicle” by Magna Electronics Inc. describes a vision system for a vehicle includes a forward facing camera configured to be disposed at a windshield of a vehicle so as to have a forward field of view through the windshield of the vehicle. The forward facing camera is operable to capture image data for an adaptive speed control system of the vehicle. U.S. Pat. No. 7,720,580B2 titled “Object detection system for vehicle” by Magna Electronics Inc. describes an imaging system which includes an imaging array sensor and a control. The image array sensor comprises a plurality of photo-sensing pixels and is positioned at the with a field of view exteriorly of the vehicle.
U.S. Pat. No. 8,446,467B2 titled “Combined speed detection, video and timing apparatus by Combined speed detection, video and timing apparatus” by TILTON; Scott K. et al. describes a system of combination speed-detection, video-recording, and timing device that can be hand-held.
US patent application US20160293002A1 titled “mobile number plate recognition and speed detection system” by EKIN; Akif describes a mobile number plate recognition and speed detection apparatus placed on the vehicles, i.e., police vehicle, with the aim of security, characterized in comprising camera which is placed at two sides of the base—front right and front left—and enables the apparatus to capture image;
US patent application US20120148105A1 titled “Automated license plate recognition system and method using human-in-the-loop based adaptive learning” by Xerox Corp. describes an automated license plate recognition (ALPR) system and method using a human-in-the-loop based adaptive learning approach. One or more images with respect to an automotive vehicle can be segmented in order to determine a license plate of the automotive vehicle within a scene. An optical character recognition (OCR) engine loaded with an OCR algorithm can be further adapted to determine a character sequence of the license plate based on a training data set. A confidence level with respect to the images can be generated in order to route a low confidence image to an operator for obtaining a human interpreted image. The parameters with respect to the OCR algorithm can be adjusted based on the human interpreted image and the actual image of the license plate. A license plate design can be then incorporated into the OCR engine in order to automate the process of recognizing the license plate with respect to the automotive vehicle in a wide range of transportation related applications.
U.S. Pat. No. 7,363,133B2 titled “Lane detection system and method” by Valeo Schalter and Sensoren GmbH describes a lane detection system for a vehicle, having a camera for sensing the carriageway markings in front of the vehicle, and an evaluation unit for evaluating the data collected by the camera.
WO patent application WO2016069253A1 titled “Augmented lane detection using kinematic data” by TRW Automotive U.S. LLC describes a system and method provided for detecting the departure of a vehicle from a set of land boundaries. A boundary determination component is configured to determine an associated set of lane boundaries for a vehicle.
WO patent application WO2007023103A1 titled “Lane Departure Warning And/or Lane keeping system” by Stephan Voltz describes a lane departure warning and/or lane keeping system for a motor vehicle comprising a sensor unit, which is oriented in the travel direction of the motor vehicle and is used for identifying traffic lane(s).
U.S. Pat. No. 8,587,649B2 titled “Lane departure warning system” by Create Electronic Optical Co Ltd describes a lane departure warning system (LDWS) installed on vehicles is revealed. The LDWS includes a camera that captures road images and the data of images is sent to an electronic control unit (ECU) for processing and recognition. The ECU is directly connected with a global positioning system (GPS) that provides vehicle speed signals so as to check whether dangerous driving occurs.
U.S. Pat. No. 9,760,806B1 titled “Method and system for vision-centric deep-learning-based road situation analysis” by TCL Research America Inc. describes a method and system for vision-centric deep-learning-based road situation analysis are provided. The method can include: receiving real-time traffic environment visual input from a camera; determining, using a ROLO engine, at least one initial region of interest from the real-time traffic environment visual input by using a CNN training method
KR patent application KR20170105845A titled “Driving assistant apparatus, distance detection method and width of lane detection method” by Hyundai Auto describes a system and method for generating a front image containing the other vehicle which is located at the front by a front camera and; And to a driving support apparatus for a control unit for the actual characters on the basis of distance information controls the speed or traveling direction of the vehicle.
EP patent application EP3143607A1 titled “Systems and methods for curb detection and pedestrian hazard assessment” by Mobileye Vision Technologies (Jerusalem) Ltd describes a system for a vehicle to identify at least one edge line candidate as an edge line of the curb.
EP patent application EP3261017A1 titled “Image processing system to detect objects of interest” by Delphi Technologies Inc. describes a method of detecting objects of interest in a vehicle image processing system using convolutional neural networks (CNNs)
WO patent application WO2017190574A1 and Chinese Patent CN105975929A, titled “Fast pedestrian detection method based on aggregation channel features” by Peking University Shenzhen Graduate School describes a fast pedestrian detection method based on aggregation channel features, which comprises an early-stage position calibration process and a later-stage position screening process. By using the technical solution of the present invention, where a training data amount is large, the classifier can automatically select features with a good recognition ability to serve as a pedestrian judgment basis.
WO patent application WO2017136578A1 titled “Partially occluded object detection using context and depth ordering” by Honda Motor Co., Ltd. describes a system and method for verifying detection of partially occluded objects (e.g., pedestrians) in the vicinity of a vehicle. An image input device captures an image and/or video of surroundings.
CN patent application CN106845430A titled “Pedestrian detection and tracking method based on accelerated area Convolutional Neural Network” by Donghua University describes a pedestrian recognition and tracking method based on an accelerated area Convolutional Neural Network where a pedestrian in an infrared image can be effectively recognized, and real-time tracking for a pedestrian target in an infrared video can be achieved.
CN patent application CN107154045A titled “Computer-vision-based method for localizing vehicle with peccancy pressing-line behavior in traffic flow diversion region” by Original Assignee Nanjing University of Aeronautics and Astronautics describes a method for localizing a vehicle with a peccancy pressing-line behavior in a flow diversion region. With the method disclosed by the invention, detection of line pressing by a vehicle in congestion can be realized
CN patent application CN107481526A titled “Detecting traffic lane change violation Records system and method for a lane change control report” by Ministry of Public Security Third Institute describes a system and method for detecting traffic lane change
CN patent application CN106935035A titled “Illegal parking vehicle real-time detection method based on SSD neural network” by Xi'an University of Electronic Science and Technology describes a vehicle real-time detection method based on an SSD neural network, and mainly solves the problems of low detection accuracy and weak robustness under the condition of complex roads and changeable illumination of weather.
CN patent application CN106373426A titled “Computer vision-based parking space and illegal lane occupying parking monitoring” method by Chengdu Tongjia Youbo Technology Co., Ltd. Describes an analysis and detection method to solve the problems of high detection cost and low efficiency of space detection and lane occupying detection in the prior art.
CN patent application CN106327878A titled “Movable illegal parking snapshot system and implementation method” by Tianjin Zhonghuan System Engineering Co., Ltd. describes a movable snapshot system and an implementation method when an behavior is established as total time of a vehicle target in an area exceeds a time threshold value preset by a system.
CN patent application CN107491753A titled “Parking violation detection method based on background modeling” by University of Electronic Science and Technology describes a detection method based on background modeling, firstly inputted video traffic monitoring dynamic background modeling, monitoring of the scene to obtain the background image;
CN patent application CN106652465A titled “Method and system for identifying abnormal driving behavior on road” by Chengdu Tongjia Youbo Technology Co., Ltd. describes a method and system for identifying an abnormal driving behavior on road through vehicle identification and a driving direction expressed by an extracted lane line to determine a vehicle driving against the traffic.
India Patent Publication 52/2017 titled “Automatic Helmet and Number Plate Detection System” provides automatic helmet and number plate detection system. This invention detects helmet and number plate on the vehicle along with the correct number plate format by taking account the vehicle type i.e. LMV, HMV or trucks. The invention uses Mega-pixel cameras placed at a distance of 50-100 m to capture the high-resolution images and extracts the number plate characters from the number plate.
Chinese Patent CN104200668 by Sichuan University titled “Image-analysis-based detection method for helmet-free motorcycle driving violation event”, discloses for helmet detection, a head top average saturation degree and an initial set value Sd, calculated and compared with each other, and a model identification technology based on a support vector machine used for identifying the license plate.
Chinese Patent CN106372662 by Tencent Technology titled “Helmet wearing detection method and device, camera, and server”, comprises: obtaining a scene video image, detecting the scene video image to obtain human position through a human position detection model built by training learning, and determining whether the human position is located in a helmet wearing area or not;
Chinese Patent CN106408000 by State Grid Corporation of China titled “Method and device for intelligent detection on personnel safety helmet”, provides a method comprises steps that a personnel sample picture of the personnel included in a first preset quantity is acquired, and a safety helmet sample picture of the personnel with a safety helmet included in a second preset quantity is acquired; DPM training of the personnel sample picture is carried out to generate a personnel distinguishing device.
Chinese Patent CN106295551 by Nanjing University Of Science And Technology titled “Worker safety helmet wearing real-time detection method based on video analysis”, based on color of the helmet and its location, where it should be.
A paper titled “Intelligent System For Helmet Detection Using Raspberry Pi”, describes a method that ensures helmet possession by a motorcyclist at all times by capturing a snapshot of the rider's helmet using Pi Camera and confirming object detection by Haar cascading technique. A paper titled “detection of motorcyclists without helmet in videos using convolutional neural network”, by C. Vishnu et al, provides an approach in which first they use adaptive background subtraction on video frames to get moving objects. Later convolutional neural network (CNN) is used to select motorcyclists among the moving objects. Again, we apply CNN on upper one fourth part for further recognition of motorcyclists driving without a helmet. The performance of the proposed approach is evaluated on two datasets, IITH_Helmet_1 contains sparse traffic and IITH_Helmet_2 contains dense traffic, respectively.
A paper titled “classification of motorcyclists not wear helmet on digital image with backpropagation neural network”, by Sutikno et al provides a system that was built is divided into two parts, namely training process and testing process. Backpropagation neural network architecture of this system consists of 400 inputs, one hidden layer consists of 40 neurons, and one output.
A paper titled “automatic Helmet Detection on Public Roads”, by Maharsh uses background subtraction and optical character recognition for fall detection and for helmet detection it uses background subtraction and Hough transform descriptor.
A paper titled “Traffic Sign Recognition with Multi-Scale Convolutional Networks” by Sermanet et. al, provides in traditional ConvNets, the output of the last stage is fed to a classifier. In the present work the outputs of all the stages are fed to the classifier. This allows the classifier to use, not just high-level features, which tend to be global, invariant, but with little precise details, but also pooled low level features, which tend to be more local, less invariant, and more accurately encode local motifs.
A paper titled “Traffic Sign Detection based on Convolutional Neural Networks”, by Y. Wu et. al., provides an approach for traffic sign detection based on Convolutional Neural Networks (CNN) by first transforming the original image into the gray scale image by using support vector machines, then use convolutional neural networks with fixed and learnable layers for detection and recognition. The fixed layer can reduce the amount of interest areas to detect, and crop the boundaries very close to the borders of traffic signs. The learnable layers can increase the accuracy of detection significantly.
A paper titled “traffic Sign Classification Using Deep Inception Based Convolutional Networks” by Mrinal Haloi, provides a deep network consists of spatial transformer layers and a modified version of inception module specifically designed for capturing local and global features together. Spatial transformer network capable of generating automatic transformation of input image is used to make classification more robust and accurate along with a modified version of GoogLeNet.
United States Patent Application US20130049988A1 by Marc Roeber et al. titled “Device and method for traffic sign recognition” provides a method and apparatus for determining whether to alert a driver of a vehicle to the presence of a traffic sign in the vehicle's driving environment.
Chinese Patent CN106022300 by INSTITUTE OF INFORMATION ENGINEERING, CAS titled “Traffic sign identifying method and traffic sign identifying system based on cascading deep learning”, provides a traffic sign identifying method and a traffic sign identifying system based on cascading deep learning. By introducing a cascading convolutional neural network idea, expanding target sign sample space, and adding more samples having supervision functions, identification of traffic signs is additionally provided with more apriori information
European Patent EP1096457B1 by Volkswagen Aktiengesellschaft titled “Method and device for electronic recognition of traffic road signs”, provides a method and a device for electronic recognition of road signs are in which the traffic signs detected with a arranged in the motor vehicle electronic camera and interpreted electronically via pattern matching method and displayed in the vehicle and/or act on the automatic speed control of the motor vehicle.
U.S. Pat. No. 6,560,529B1 by Holger Janssen titled “Method and device for traffic sign recognition and navigation”, provides a method and a coupled system for road sign recognition and for navigation is proposed, which enables a bidirectional data transmission between the road sign recognition device and the navigation device.
U.S. Pat. No. 8,983,136B2 by Ricoh Company, Ltd. titled “Traffic sign detecting method and traffic sign detecting device”, provides a method and a device for detecting traffic signs in an input image camera. The method comprises a color space converting step of converting the input image into a HSV color space image; a filtering step of filtering, based on a predetermined pass range of a standard color of each of the traffic signs, the HSV color space image to obtain a filtered image, and then generating one or more connected domains based on one or more regions in the filtered image;
United States Patent Publication US20110109476A1 by Mitsubishi Electric Research Laboratories Inc. titled “Method for Recognizing Traffic Sign”, provides a method recognizes a set of traffic signs in a sequence of images acquired of a vehicle environment by a camera mounted in a moving vehicle by detecting in each image, a region of interest (ROI) using a parameter space transform. “same” or “different.” This enables construction of an efficient multi-class classifier.
A paper titled “License Plate Detection and Recognition Using Deeply Learned Convolutional Neural Networks”, by Computer Vision Lab, Sighthound Inc., Winter Park, Fla., provides an end-to-end license plate detection and recognition method using novel deep CNNs. They claim it to be not only computationally inexpensive, but also it outperforms competitive methods on several benchmarks after conducting successful experiments on leading benchmarks.
A paper titled “Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs” by Hui Li, Chunhua Shen, tackles the problem of vehicle license plate detection and recognition in natural scene images using deep neural networks and LSTMs to learn high-level features in a cascade framework, which lead to improved performance on both detection-recognition. Firstly, they train a 37-class convolutional neural network (CNN) to detect all characters in an image, which results in a high recall, compared with conventional approaches such as training a binary text/non-text classifier. False positives are then eliminated by the second plate/non-plate CNN classifier. The paper “Vehicular Number Plate Recognition Using Edge Detection and Characteristic Analysis of National Number Plates”, proposes a framework that uses a camera installed at roadside to detect the vehicle number plate. They use Sobel edge detection for plate localization, template matching and fuzzy logic for recognition. They make use of the characteristics of the vehicle number sequences to further enhance performance.
Chinese Patent CN104298976 by University Of Electronic Science And Technology Of China titled “License plate detection method based on convolutional neural network”, provides license plate images under different conditions can be accurately recognized through the Adaboost license plate detector based on the Haar characteristics and the convolutional neural network complete license plate recognition model, meanwhile, characters are segmented through the multi-threshold segmentation algorithm, character images can be more easily and conveniently segmented.
Chinese Patent CN106250892 by Wan Yongxiu titled “Automatic recognition method for license plate characters”, provides the following steps of inputting a colorful vehicle image; obtaining current light intensity; if the current light intensity is higher than preset light intensity, adopting a license plate location algorithm based on color point pair search and mathematical morphology to carry out license plate location on the colorful vehicle image to obtain a license plate image.
Chinese Patent Publication CN106845480 titled “Method for recognizing license plate from picture”, provides a method, the license plate positioning is realized through combination of color positioning and Sobel positioning; through carrying out color judgment on the picture of the license plate, carrying out binaryzation and extracting a character outline and a circumscribed rectangle, the character segmentation is realized; and the characters of the license plate are recognized through a neural network algorithm.
Chinese Patent CN106874907 by Beijing Haidian Branch Of Bocom Intelligent Information Technology Co., Ltd. titled “License plate recognition model establishing method and device”, provides a method by which The license plate images are enlarged to a preset size to acquire license plate image samples; a number of license plate image samples and license plate information in the license plate image samples are used as training data to train a neural network model until the recognition rate of the neural network model for the license plate information of the license plate image samples is greater than a preset threshold or the loss value of the loss function of the neural network model converges to a preset value; and the trained neural network model is used to recognize the license plate information of a license plate image to be recognized.
United States Patent Publication 20130311075A1 filed by Continental Automotive Systems, Inc. teaches a safety system for motorcycle comp-rises at least one sensor mounted to the motorcycle to sense a feature of an environment surrounding the vehicle. An electronic control unit is configured to receive a signal from the at least one sensor and determine a probability that a safety incident may occur based upon the at least one feature.
U.S. Pat. No. 9,704,060B2 by Feiyue Wang titled “Method for detecting traffic violation”, relates to a technical field of traffic monitoring, and more particularly to a method for detecting traffic violation. The present invention includes firstly localizing vehicle salient parts through salient features including vehicle license numbers and vehicle rear lights, and representing a vehicle with the vehicle salient parts, then tracking the vehicle with a Kalman filter based on the vehicle salient parts, and finally detecting vehicle violation through moving trajectory analysis and setting violating detecting areas.
Indian Patent 220469 filed by Diagaraj. R for “Side Stand Detection” relates to the field of automobiles industry, especially for two-wheeler vehicles using side stand apart from the Main center stand provided therein for the resting of the vehicle. The invention in particular provides an electronic circuit system which alarms or sirens the person handling the vehicle about the unreleased side stand and there by prevent him from being endanger or to have safe movement. Thus the present invention provides an alarm system for the vehicle handler or rider about the ignorance to release the side stand under vehicle movement.
U.S. patent application Ser. No. 15/689,350, titled “On-demand Roadway Stewardship System”, filed by Jayant Ratti, provides an on-demand roadway stewardship system with video reporting features is disclosed. The invention described herein is comprised of a system that allows users with mobile device cameras to record and report roadway safety incidents, traffic violations, crimes and infrastructure problem. Users are encouraged to become stewards by engaging in the system's rewards program. An on-demand style cloud infrastructure is presented which speeds up video processing and citations. Objects of the invention are to enhance safety and increase public participation in safety.